The invention relates to a method and a device for displaying at least part of a person""s body with a modified appearance by providing first image data representative of the unmodified appearance of said body part, second image data representative of a modification in the appearance of said body part and position data representative of the position and spatial orientation of said body part, wherein said first and second image data are linked to produce and to display third image data representative of said modified appearance of said body part, in dependence on said position data, such that said displayed modified appearance follows changes in the position of said body part in a dynamic
Many clients of hairdressers, beauticians, jewellers, dentists, opticians or boutiques have a need for support and advice in the selection of their purchases-or treatment. A person""s imaginative powers continue to play an important role in this respect. Not everybody possesses the same talents in this regard and consequently it is attractive to be more sure of the result in advance by being given an adequate personal visualisation thereof. This applies in particular in those cases where the result cannot be immediately undone, as may for example be the case with a hairstyle. The decision to take a particular treatment and the selection of a particular type of treatment can be made more efficiently when said decisions are supported by such a visualisation, as a result of which the time involved in said treatment can be reduced.
U.S. Pat. No. 4,823,285 discloses a method for representing a person with a modified hairstyle by means of a computer, a television camera and a viewing screen. A static video image of the respective person""s head is made, which is displayed on the screen. After a new hairstyle has been selected from a data file, the original hairstyle is substituted on the screen for the selected new hairstyle.
U.S. Pat. No. 5,404,426 discloses a method and a device for visualising hairstyles, using graphic computer techniques. Unlike the presentation of static hairstyles as disclosed in the aforesaid U.S. Pat. No. 4,823,285, a system is disclosed by means of which the hairdo can be modified interactively on the screen. To this end a computing model is used, by means of which modifications in an artificially formed hairstyle resulting from external forces are represented. Said external forces, among which the influence of gravity or the forces resulting from the modelling of a hairdo by means of a comb or the like, can be simulated via a computer mouse or a computer keyboard.
It has become apparent that such a visualisation will only have the intended supporting effect in selecting a purchase or a treatment if said visualisation can appeal to a person""s intuitive behaviour, wherein perception plays an important role. Said perception is considerably enhanced by movement.
U.S. Pat. No. 4,539,585 discloses a previewer for the dynamic viewing of article desired to be worn by a user in a complete series of poses without actually trying on the article. In an embodiment, the user creates in real time a series of poses which are time-framed and position orientation coded. The image of the article is computer generated and re-imaged in accordance with the coded position information of the user. The re-imaged article image and the user video image are than video mixed, optically mixed or superimposed in digital image storage memory, such that a composite image may be viewed by the user in a dynamic viewing mode.
Unlike the above-described prior art, this previewer makes it possible to present the modified appearance of a user in a dynamical manner, such that the person in question can evaluate the effect of the modified appearance in several positions thereof. However, because the position coding depends on coded indicia segments, which segments in toto form a composite image of the article, only a limited number of poses can be actually covered.
U.S. Pat. No. 5,515,268 discloses a previewer essentially of the type disclosed by U.S. Pat. No. 4,539,585, however enhanced with shape sensor means.
Although these known previewers appeal more to a person""s intuitive behaviour than the static presentations, in practice, there is a need for a more sophisticated presentation technique such that a modified appearance of at least part of the human body can be displayed in a true-to-life manner, as if the person is looking at himself in a mirror with the modified body part.
Accordingly, it is an object of the invention to provide an improved technique for displaying a body part with a modified appearance, which enables the person in question to evaluate the effect of the modified appearance in a more true-to-life manner.
According to the invention for linking said first and second image data a virtual image data space is created comprising an aspect or aspects of said appearance of said body part to be modified, wherein said second image data comprise virtual image data elements, and wherein said modification of said appearance of said body part is provided by modifying said virtual image data space using virtual image data elements of said second image data, wherein said virtual image data space thus formed and said first image data not comprised thereof are linked to produce said third image data in dependence on said position data.
The term virtual image data space is to be construed in that a real image data space of the first image data representative of the unmodified appearance of the body part and comprising the aspect or aspects of the body part to be modified, is virtually reconstructed. Next, the actual modification is carried out by modifying the virtual image data space using second image data comprised of virtual image data representative of the intended modification. The modified appearance is than created by suitable linking of the modified virtual image data space and the first image data, providing third image data representing the modified appearence of the body part.
The virtual image data space concept according to the invention is essentially able to cope with any changes in the position and spatial orientation of the body part with respect to its environment, such as shading due to differences in the incident light, hiding of portions of the environment due to movement of the body part, etc. This, because the influences of such changes on the appearance to be displayed are virtually constricted in the virtual image data space comprising the modified aspect or aspects of the appearance using respective virtual image data elements.
Unlike the above-described prior art, the invention makes it possible to present the modified appearance in a more dynamical and true-to-life manner, such that the person in question can evaluate the effect of the modified appearance in essentially unrestricted positions thereof. It will be appreciated that such a dynamic, interactive representation is a more efficient support of a person""s imagination, as a result of a stronger true-to-life perception. That is, the modified appearance directly follows the changes in the position of the body part in a true-to-life manner, as if the person is looking at it in a front mirror with the body part thus modified.
The method according to the invention provides to select patterns or models in connection with a desired modification, which patterns or models may be selected from a data file, if desired. The actual modified appearance is than created in accordance with the patterns or models selected and composed of virtual image data elements.
Selection of a respective pattern or model may take place in several manners, for example by making a selection from a pattern or model book, whether or not electronically, whereby the appearance of a respective body part is shown commensurately with a respective pattern or model.
The selection of a particular pattern or model may be provided in an efficient manner by first displaying, in accordance with another embodiment of the method according to the invention, the respective part of the person""s body in a fixed position with a modified appearance commensurate with a pattern or model to be selected. That is, the body part in question is displayed in a particular position, such as a reference position, with a modified appearance, so that the person in question is given a first indication of the influence of a pattern or model on the respective part of his own body. The patterns or models that have been selected in this manner can then be used for visualisation in a dynamic and true-to-life manner, as described above.
It has furthermore become apparent that the selection of a particular pattern or model can also be supported by displaying further information in combination with the modified appearance, which further information may be visual information or auditive information, or both.
In this connection, further information directly relating to the modified appearance, such as information with regard to care, cost, possible risks and the like may be considered, but for example also information which places the person in question with his modified appearance in a desired situation, a decor, for example, but also clothes and the like. In accordance with one embodiment of the method according to the invention, said further information may be provided in the form of (moving) images, such as a video clip. In this way it is also possible to initiate exchange of information by means of questions and/or answers.
Besides the representation of the modified appearance commensurate with a selected pattern or model, the invention, in a preferred embodiment thereof, furthermore provides a possibility to modify one or more aspects of the modified appearance interactively during said representation. A modification in the colour, the volume, the length and the like can be considered.
The technique according to the invention is not limited to one or more specific parts of a person""s body, actually, any desired body modification can be simulated in a manner which optimally matches a person""s perception, so as to support said person""s selection of a respective modification.
The invention is especially applicable as regards a person""s head, wherein the modified appearance comprises modifications in one or more aspects of the hairstyle, the make-up, accessories such as glasses, coloured contact lenses, earrings, piercings and the like, and also modifications of the nose, mouth, lips, teeth, eyes, ears, eyebrows, moustache, beard and other facial features, among which the effect of a facelift or other reconstructive work, for example, wherein for each aspect to be modified a separate virtual image data space is created or wherein aspects to be modified are comprised by a common virtual image data space. The modifications as such, i.e. the hair, nose, etc. are formed from respective virtual image data elements composed of polygons, texture etc., for example.
The first and the second image data can be linked in various manners, according to an embodiment of the method of the invention, for example, this is provided by first arranging the second image data in a relative position commensurate with the first image data. The second image data can be selected from a plurality of pre-stored virtual image data elements in a data file, or be computed by means of suitable mathematical algorithms, on the basis of the position data as address information.
In the case of a selection from pre-stored second image data comprising data recorded from various angles and positions, those image data are selected therefrom during representation that correspond most closely as regards the recording angle or position with the actual angle or position of the respective body part. Next, the image data for the exact angle and position can then be derived by means of an interpolation method. Mathematical techniques for computing such views comprise, inter alia, descriptions including one or more characteristic parameters and polygons.
In a preferred embodiment of the method according to the invention, the virtual image data space is created relative to a generic type of said body part, wherein data representative of the dimensions and geometry of said body part are generated from said first image data, on the basis of which data said generic body type is scaled into commensuration with said dimensions and geometry of said body part according to said first image data.
The advantage of this manner is that the virtual data space can be directly generated from the correct viewing direction, as a result of which it can be adapted relatively easily and fast to the movements of the body part.
In a yet further emobidment of the invention, the second image data are scaled into commensuration with said scaled generic body type, and said virtual image data space is modified by said scaled second image data.
The image data are linked in such a manner that portions of the respective body part which will not be visible in the modified appearance are not shown. For example, assuming that the person in question wears glasses and wishes to see how he looks with a new frame, the data relating to the current glasses of the person in question will have to be removed from the first image data. It should be considered thereby that the person in question will necessarily have to use his glasses in order to be able to view the modified appearance.
Another problem which plays a role in this connection is known as occlusion. When the first and the second image data are being linked, it is taken into account that details of the body part in question, which in reality are hidden behind a selected pattern or model, and which are not visible, therefore, must not be shown in the modified appearance being displayed, either.
On the other hand, a particular modification, for example a different hairstyle, may make it necessary to show parts of a person""s face which are not provided by the image data which are representative of the unmodified appearance of that person. In such a case, extrapolation of image data is required by adding respective virtual image data elements. Such as derived from the generic body type.
Both problems can be solved, according to the invention, by making the scaled generic body type data transparant and overlaying same with the third data before display thereof.
In a preferred embodiment of the invention, tracking means are coupled with the body part in question, for example one or more so-called xe2x80x9ctrackersxe2x80x9d, which provide position data relating to the position and the spatial orientation (angle) of the body part in question. It will be appreciated that according to the invention said tracking means, insofar as they are visible in the first image data, must not be visible in the third image data being displayed.
In order to provide an optimally true-to-life representation or simulation of the modified appearance, the image data are preferably recorded and displayed as video data with a standard picture frequency of 25 pictures/second or higher.
Unlike the prior art discussed above, the virtual image data according to the invention provides for three-dimensional processing, wherein the modified appearance can be displayed both two-dimensional or three-dimensional. Three-dimensional processing is very advantageous for providing the desired true-to-life visualization according to the present invention.
The invention also provides a device for displaying at least one part of a person""s body with a modified appearance thereof as discussed above, which device comprises first means for providing first image data representative of the unmodified appearance of the body part, second means for providing second image data representative of a modification in the appearance of the body part, position determining means for providing position data representative of the position and spatial orientation of the body part, processing means for electronically processing the first and second image data in dependence on the position data, to produce third image data representative of the modified appearance of the body part, and means for displaying the third image data characterized in that the second means are arranged for providing second image data having the form of virtual image data elements, the processing means are arranged for creating a virtual image data space comprising an aspect of the body part to be modified, for adding to the virtual image data space virtual image data elements and for linking the virtual image data space thus formed and the first image data not comprised by the image data space to produce third image data, and for displaying same on the display means.
In a preferred embodiment of the device according to the invention, the recording means comprise at least one fixedly disposed video camera, and the tracking means comprise one or more trackers for determining the position and the spatial orientation of the respective body part relative to the recording means.
It is noted that the video camera may also be movably disposed for the purpose of the invention. Furthermore, several video cameras may be used for recording, and displaying the body part from various angles, if required, in particular for use with three-dimensional processing and relating to the generic type body part disclosed above.
The invention in particular provides a device comprising a semi-transparent mirror, wherein the recording means are disposed on one side of the semi-transparent mirror, whilst the display means are disposed on the other side thereof, in such a manner that the image being displayed by the display means can be viewed from the respective other side of the semi-transparent mirror.
With a device according to the invention which is configured in this manner, the person in question looks in a mirror, as it were, whereby he sees the body part with the modified appearance instead of the actual body part. Let us assume a situation wherein the body part in question is a person""s face. In that case the recording means and the display means will be arranged in such a manner with respect to the semitransparent mirror, that said person directly looks in the recording means via the mirror. This creates an illusion of the person in question looking in a real mirror, and all the movements he makes in front of the mirror are directly recorded and processed by the recording means, that is, the video camera(s).
In a preferred embodiment of the device according to the invention, means are provided for storing data comprising one or more patterns or models which are representative of an intended modification in the appearance of a respective body part. The data in question may be video data, that is, data which are directly suitable for being displayed (xe2x80x9cbitmapxe2x80x9d) or data which can be processed into video data, like virtual image data elements.
The storage means may consist of a read-only memory, for example in the form of a smart card, a magnetic card or other information carrier, such as a diskette or the like, which can be coupled to the processing means of the device according to the invention in a manner which is known per se.
In yet another embodiment of the device according to the invention, the storage means may also be disposed at a remote location, whereby the device and the storage means are provided with transmission means for the remote exchange of the respective image data and control data. The transmission means may for example consist of a suitably programmed modem and communication equipment for coupling of the storage means and the device via the public telecommunications network, but also via the Internet, for example. It will be appreciated that this is a highly interesting aspect of the invention, because this makes it possible to continually adapt the applications of the device to the prevailing views on fashion and the like. Furthermore this enables the remote installation of new software in the device.
In an embodiment of the device according to the invention, the storage means are arranged in such a manner that position data and/or image identification or selection data are linked to the image data.
In yet another embodiment of the device according to the invention, modifications in one or more aspects of the modified appearance of a body part being displayed can be made in an interactive manner by input means which are coupled to the processing means. In this connection, the use of a computer mouse, a keyboard, touch screen and the like for modifying the colour, the volume and the like of a modified appearance may be considered. In an advantageous embodiment, this option is controlled intuitively, in such a manner that a user having some experience in the control of computer systems can operate the device himself without constant guidance.
The device according to the invention is furthermore arranged for storing a selected modified appearance by providing a so-called hard copy, for example, by means of a (video) printer or a soft copy on a diskette, a smart card or other information carrier suitable for registering (video) data. Said prints may furthermore include further information, such as commercial information or information on costs, care, etc.
The device according to the invention may furthermore be provided with means for presenting further visual and/or auditive information in combination with or in support of the body part with the modified appearance being displayed a ready explained before.